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REMARKS 

Reexamination and reconsideration of this application as amended is requested. Claims 
1, 12, 18, and 29 have been amended. After this amendment, Claims 1-29 remain pending in this 
application. Applicant submits that the present response places the application in condition for 
allowance. Entry of the present response is therefore respectfully requested. 

Telephonic Interview 

As an initial matter, the Applicant would like to thank Examiner Tucker for the 
telephonic interview held on Monday, January 7, 2008. Patent attorney Thomas Grzesik and 
Examiner Tucker participated in the telephone call. Discussed was a general overview of the 
present invention and how it differs from the prior art of record. The Applicant agreed to further 
clarify the claim language to positively recite that the three or more images that are captured 
include at least an overlapping portion of the scene relative to each other and to further clarify. 
No further agreement was reached on the telephonic interview. 



Claim Rejections - 35 USC § 101 

The Examiner rejected Claim 29 under 35 U.S.C. § 101 stating that Claim 29 is directed 
towards non-statutory subject matter. In particulat the Examiner stated that "claim 29 dicusses 
recovering depth information in the preamble of the claim as the main purpose of the method, 
but does not specifically connect depth information to a tangible concrete result". The Applicant 
has amended Claim 29 to further recite: 

A method of recovering depth information for pixels of a base image representing 
a view of a scene, the method comprising the steps of: 

capturing a base image representing a view of a scene; 
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tracing at least one parameter surface associated with the base image, each 
of the at least one parameter surface traced starting from at least one 
predetermined seed pixel point associated with the base image; and 

calculating a derivative of function E(g) with respect to parameter g by 
using finite difference to minimize the following equation 

E(g) = ^{l-NCC[l i (u(glv(g)),I 0 {u ] ,v ] )^, 

where NCC[/ i .(k,vUo(k/.v / ) J is a normalized cross-correlation between / ; (w,v) 
and I 0 (Uj,Vj) ; 

l i (u, v) is a first window of size coxco centered at pixel (u, v) in I t , 
I 0 (Uj,Vj) is a second window of size coxco centered at pixel (uj,Vj) in I 0 , 

Ii is a reference image, 

Io is a base image. 

(u, v) is a pixel point closest to Pi(Co+gL(uj, vj)); 

Co+gL(uj, vj) is a 3-D point that projects to (uj, vj) in the base view, and to 

Pi(C 0 +gL(uj, Vj)) in the i reference view; 

Co is the base image camera's center of projection; 

L(uj,Vj) is the unit vector from C 0 to the point on the image plane that corresponds 

the pixel (uj,vj); 

g is a depth parameter; and 

storing the calculated derivative of the function E(g) in a memory!"!"-!! ; and 
generating a three-dimensional image of at least a portion of the scene 

using at least the calculated derivative of the function E(g) that has been stored in 

memory. 

Support for this amendment can be found in the Specification as originally filed at, for 
example, page 17, lines 5-16, page 23, lines 15-20; and page 26, lines 1-13. No new matter has 
been added. 



The Applicant respectfully submits that "... generating a three-dimensional image of at 
least a portion of the scene using at least the calculated derivative of the function E(g) that has 
been stored in memory " as now recited for amended Claim 29 produces a "useful, concrete and 
tangile result". For example, the generation of a three-dimensional image using at least the 
calculated derivative of the function E(g) that has been stored in memory shows results in a 
manipulation of the data outside the computer to produce a three-dimensional image. Therefore, 
the Applicant suggests that the rejection of Claim 29 under 35 U.S.C. § 101 has been overcome 
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and should be withdrawn. 



Claim Rejections - 35 USC § 103 

The Examiner rejected Claims 1, 4-9, 12-18, and 21-28 under 35 U.S.C. 102(b) as being 
unpatentable over Chen and Medioni, "A Volumetric Stereo Matching Method: Application to 
Image-Based Modeling", IEEE 1999, (hereinafter "Chen [VSMM99]"), and U.S. Patent No. 
5,853,672 to Lu (hereinafter "Lu"). 

Applicant has amended independent Claims 1, 12, and 18 to more clearly and distinctly 
recite the presently claimed invention. Applicant has amended Claims 1, 12 and 18 to more 
clearly recite 

detecting a plurality of pixels in a base image that represents a first view 
of a scene, wherein the first view is one of at least three relative views of a scene; 

determining 3-D depth of the plurality of pixels in the base image by 
contemporaneously matching image features between three or more relative 
images of a scene by matching correspondence to a plurality of pixels in three or 
more relative images representing the at least three relative views of the scene, 
wherein the three or more images are captured relative to each other and are of the 
same scene, and wherein each of the three or more images includes at least an 
overlapping portion of the scene relative to each other, wherein the at least three 
relative views of the scene are situated in a non-linear arrangement relative to 
each other and are further oriented in non-parallel planes relative to each other; 
and 

tracing pixels in a virtual piecewise continuous depth surface by spatial 
propagation starting from the detected pixels in the base image by using the 
matching and corresponding plurality of pixels in the three or more relative 
images to create the virtual piecewise continuous depth surface viewed from the 
base image, each successfully traced pixel being associated with a depth in the 
scene viewed from the base image. 

Support for these amendments may be found in the Specification as originally filed, see 
for example FIG. 6; page 12, lines 1-3 ("M-View Robust Matching"); page 22, lines 1-22; page 
23, lines 1-21; page 26, lines 3-13; and page 27, lines 9-12.. No new matter was added. 
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The Examiner states on page 4 of the present Office Action, that Chen [VSMM99] 
teaches "at least three relative views of a scene wherein three of more images are captured 
relative to each other and are of the same scene". In particular, the Examiner states that "[t]he 
views are all of the same object and the views are 'relative' to each other. What views of the 
same object would not be 'relative'?". The Applicant respectfully disagrees with the Examiner 
and has amended the independent claims to more clearly recite "determining 3-D depth of the 
plurality of pixels in the base image by contemporaneously matching image features between 
three or more relative images of a scene by matching correspondence to a plurality of pixels in 
three or more relative images representing the at least three relative views of the scene, wherein 
the three or more images are captured relative to each other and are of the same scene, and 
wherein each of the three or more images includes at least an overlapping portion of the scene 
relative to each other ". 

As has been extensively argued, Chen [VSMM99] only teaches stereo images, that is, Chen 
[VSMM99] only teaches capturing two images at a time and matching two images at a time. See 
Chen [VSMM99] generally and at section 3.2 "Disparity surface extraction". 

The present invention, on the other hand, now more clearly recites: 
[•••] 

determining 3-D depth of the plurality of pixels in the base image by 
contemporaneously matching image features between three or more relative 
images of a scene by matching correspondence to a plurality of pixels in three or 
more relative images representing the at least three relative views of the scene, 
wherein the three or more images are captured relative to each other and are of the 
same scene, and wherein each of the three or more images includes at least an 
overlapping portion of the scene relative to each other 

[•••] 

In other words, robust matching is performed by contemporaneously matching image 
features between three or more relative images of a scene as compared to stereo matching as 
taught by Chen [VSMM99] . Chen [VSMM99] is merely teaching matching two images at a 
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time. Also, each of the three or more views of the presently claimed invention is relative to one 
another, e.g., they all include at least an overlapping portion of the scene relative to each other. 

As stated in the two previous Responses, Chen [VSMM99] only teaches using two 
cameras to capture two images and processing these images two at a time. Therefore, Chen 
[VSMM99] matches correspondence between pixels of two images at a time and then obtains a 
result. The (u,v,d) space of Chen [VSMM99] is working with only two images at a time. The 
present invention, on the other hand, matches correspondence between all of the images (e.g., 
three or more) captured relative to each other and that include at least an overlapping portion of 
the scene relative to each other , and obtains a result. Therefore, the present invention 
distinguishes over Chen [VSMM99] for at least these reasons. 

The Applicants respectfully point out to the Examiner that Chen [VS99MM] clearly 
teaches that the views of the scene §rg situated in a co-linear arrangement. As discussed above, 
Chen [VSMM99] teaches stereo images, which by definition uses two cameras. As shown in 
Example 1 and Example 2 above, and in FIG. 1 of the specification as originally filed, a straight 
line exists between two points. Even though the object in Chen [VSMM99] can rotate, the 
cameras and the views captured by the cameras remain co-linear, as shown by Example 2. 
Therefore, because Chen [VSMM99] only teaches using two cameras, Chen [VSMM99] teaches 
that the views of the scene are situated in a co-linear arrangement. On the other hand, the 
presently claimed invention recites "the at least three relative views of the scene are situated in a 
non-linear arrangement relative to each other and are further oriented in non-parallel planes 
relative to each other" as shown in Example 3 above and FIG. 3 of the specification as originally 
filed. As can be seen from Example 3, each of the three (not limited to three) cameras captures a 
view that is relative to a view of the three another camera. Accordingly, the present invention 
distinguishes over Chen [VSMM99] for at least these reasons as well. 

The Examiner correctly states that Chen [VSMM99] "does not explicitly disclose 
wherein the three images are captured simultaneously". However, the Examiner goes on to 
combine Chen [VSMM99] with Lu to overcome the deficiencies of Chen [VSMM99] . The 
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Applicant respectfully points out that the claim language no longer recites "wherein the three 
images are captured simultaneously". However, Lu also does not teach or suggest: 
[•••] 

determining 3-D depth of the plurality of pixels in the base image by 
contemporaneously matching image features between three or more relative 
images of a scene by matching correspondence to a plurality of pixels in three or 
more relative images representing the at least three relative views of the scene, 
wherein the three or more images are captured relative to each other and are of the 
same scene, and wherein each of the three or more images includes at least an 
overlapping portion of the scene relative to each other 

[•••] 

Lu is directed towards three-dimensional imaging system for moving objects that can use 
multiple sets of two video cameras. The Examiner directs the Applicant to col. 3, lines 56-64 
and col. 6, lines 1-10 of Lu. However, these citations of Lu merely state that a plurality of 
images are simultaneously captured from various cameras. Lu teaches that three imaging heads 
surround an object of interest every 120 degrees. See FIG. 1 of Lu. Each imaging head includes 
a light projector that is placed in between two video cameras. See col. 3, lines 39-47. The light 
projector is used to project line strips onto the object of interest. Lu explicitly teaches that the 
two cameras in an imaging head are separated by a base line and are calibrated with respect to a 
fixed relative position and focus. See col. 3, lines 45-47 and lines 66-67. Therefore, the cameras 
in an imaging head are co-linear and the present invention distinguishes over Lu for at least this 
reason. 

Furthermore, Lu (similar to Chen [VSMM99]) only teaches matching two images at a 
time. As stated in the previous Response, each imaging head of Lu captures two images and 
even though up to six images can be captured at once, only the two separate images from each 
imaging head are matched with each other. Nowhere does Lu teach or suggest 
". . . contemporaneously matching image features between three or more relative images of a 
scene..." Accordingly, the presently claimed invention distinguishes over Lu for at least these 
reasons as well. 
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For the foregoing reasons, Claims 1, 12, and 18 distinguish over Chen [VSMM99] in 
view of Lu. Claims 4-9, 13-17, and 21-28 depend from Claims 1, 12, and 18, respectively, and 
since dependent claims contain all the limitations of the independent claims, claims 4-9, 13-17, 
and 21-26 distinguish over Chen [VSMM99] alone or in combination with Lu, as well. 
Accordingly, Applicant believes that the rejection under 35 U.S.C. § 103(a) has been overcome 
and respectfully requests that this rejection be withdrawn. Additionally, neither single reference 
nor a combination therefore teaches or suggest the presently claimed invention. 

The Examiner rejected Claims 27-28 under 35 U.S.C. 103(a) as being unpatentable over 
Chen [VSMM99] . 

Claims 27-28 depend from amended Claim 18. The above arguments and remarks 
regarding Claim 18 are likewise applicable here in support of the allowability of Claims 27-28. 
These applicable arguments have already been presented above and will not be repeated here. 
Nowhere does Chen [VSMM99] teach or suggest: 
[•••] 

determining 3-D depth of the plurality of pixels in the base image by 
contemporaneously matching image features between three or more relative 
images of a scene by matching correspondence to a plurality of pixels in three or 
more relative images representing the at least three relative views of the scene, 
wherein the three or more images are captured relative to each other and are of the 
same scene, and wherein each of the three or more images includes at least an 
overlapping portion of the scene relative to each other 

[•••] 

Therefore, Applicant believes that the rejection of Claims 27-28 has been overcome and 
requests that the rejection be withdrawn. 

The Examiner rejected Claims 2-3 and 19-20 under 35 U.S.C. 103(a) as being 
unpatentable over Chen [VSMM99], and U.S. Patent No. 5,852,672 to Lu, in view of Zhang, 
Deriche, Faugeras, and Luong, "A Robust Technique for Matching Two Uncalibrated Images 
Through the Recovery of Unknown Epipolar Geometry", INRIA 1994), (hereinafter "Zhang"). 
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Claims 2-3 and 19-20 depend from amended Claims 1 and 18, respectively. The above 
arguments and remarks regarding Claims 1 and 18 are likewise applicable here in support of the 
allowability of Claims 2-3 and 19-20. These applicable arguments have already been presented 
above and will not be repeated here. With respect to Zhang, the presently claimed invention also 
distinguishes over Zhang for the reasons stated in the previous filed Responses. Nowhere does 
Chen [VSMM99], Lu, and Zhang, taken individually or in any combination thereof teaches or 
suggests: 

[•••] 

determining 3-D depth of the plurality of pixels in the base image by 
contemporaneously matching image features between three or more relative 
images of a scene by matching correspondence to a plurality of pixels in three or 
more relative images representing the at least three relative views of the scene, 
wherein the three or more images are captured relative to each other and are of the 
same scene, and wherein each of the three or more images includes at least an 
overlapping portion of the scene relative to each other 

[•••] 

Applicant believes that the rejection of Claims 2-3 and 19-20 under 35 U.S.C. 103(a) has 
been overcome. The Examiner should withdraw the rejection of these claims. 

The Examiner rejected Claims 10-11 under 35 U.S.C. 103(a) as being unpatentable over 
Chen [VSMM99] and U.S. Patent No. 5,852,672 to Lu, in view of Okutomi and Kanade, "A 
Multiple-Baseline Stereo", IEEE 1993, (hereinafter "Okutomi"), in further view of Lewis, "Fast 
normalized Cross-Correlation", 1995, (hereinafter "Lewis"). 

Claims 10-11 depend from amended Claim 1. The above arguments and remarks 
regarding Claim 1 are likewise applicable here in support of the allowability of Claims 10-11. 
These applicable arguments have already been presented above and will not be repeated here. 
Neither Chen [VSMM99], Lu, Okutomi, or Lewis, taken individually or in any combination 
thereof, teaches or suggests: 
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[•••] 

determining 3-D depth of the plurality of pixels in the base image by 
contemporaneously matching image features between three or more relative 
images of a scene by matching correspondence to a plurality of pixels in three or 
more relative images representing the at least three relative views of the scene, 
wherein the three or more images are captured relative to each other and are of the 
same scene, and wherein each of the three or more images includes at least an 
overlapping portion of the scene relative to each other 

[•••] 

Accordingly, the Applicant believes that the rejection of Claims 10-11 under 35 U.S.C. 
103(a) has been overcome. The Examiner should withdraw the rejection of these claims. 

Conclusion 

The foregoing is submitted as full and complete response to the Official Action mailed 
November 14, 2007, and it is submitted that Claims 1-29 are in condition for allowance. 
Reconsideration of the rejection is requested. Allowance of Claims 1-29 is earnestly solicited. 

No amendment made was related to the statutory requirements of patentability unless 
expressly stated herein. No amendment made was for the purpose of narrowing the scope of any 
claim, unless Applicant has argued herein that such amendment was made to distinguish over a 
particular reference or combination of references. 

Applicant acknowledges the continuing duty of candor and good faith to disclose 
information known to be material to the examination of this application. In accordance with 37 
CFR § 1.56, all such information is dutifully made of record. The foreseeable equivalents of any 
territory surrendered by amendment are limited to the territory taught by the information of 
record. No other territory afforded by the doctrine of equivalents is knowingly surrendered and 
everything else is unforeseeable at the time of this amendment by the Applicant and the 
attorneys. 



Page 19 of 20 



Appl. No. 09/927,558 

Reply to the Office Action of November 14, 2007 



The present application, after entry of this amendment, comprises twenty-nine (29) 
claims, including four (4) independent claims. Applicant has previously paid for twenty-nine 
(29) claims including four (4) independent claims. Applicant, therefore, believes that a fee for 
claims amendment is currently not due. 

If the Examiner believes that there are any informalities that can be corrected by 
Examiner's amendment, or that in any way it would help expedite the prosecution of the 
patent application, a telephone call to the undersigned at (561) 989-9811 is respectfully 
solicited. 

The Commissioner is hereby authorized to charge any fees that may be required or credit 
any overpayment to Deposit Account 50-1556. 

In view of the preceding discussion, it is submitted that the claims are in condition for 
allowance. Reconsideration and re-examination is requested. 

Respectfully submitted, 

Date: 2/14/2008 By: /Jose Gutman/ 

Jose Gutman 
Reg. No. 35,171 

FLEIT, KAIN, GIBBONS, GUTMAN 
BONGINI & BIANCO P.L. 
551 N.W. 77th Street, Suite 111 
Boca Raton, FL 33487 
Tel (561) 989-9811 
Fax (561) 989-9812 
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